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The No Child Left Behind Act of 2001 



As a reauthorization of the Elementary and Secondary Education Act (ESEA), 
Congress passed the No Child Left Behind (NCLB) Act of 2001. Signed into law 
by President Bush in January 2002 , the legislation brings many significant changes 
to schools nationwide . 

s a result of the NCLB Act, schools and districts are seeking 



scientifically based research (SBR). The purpose of this brochure is to 
help administrators, educators, parents, and community members understand 
the applicability of SBR and its components as they relate to school 
improvement efforts under NCLB legislation. It is designed to help you 
understand how to identify SBR, evaluate evidence of effectiveness, ask 
key questions about the research you find, and apply SBR to your school 
improvement efforts. 

According to the NCLB Act (2002), a local educational agency or consortium can use 
federal funds to enact an improvement effort that: 

“Has been found, through scientifically based research to significantly improve the 
academic achievement of students participating in such program as compared to 
students in schools who have not participated in such program; or has been found 
to have strong evidence that such program will significantly improve the academic 
achievement of participating students” [Section 1606a, 11A-B]. 
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ii information to identify, plan, and implement federally funded 
programs and practices that have been proven to be effective through 



Evidence of Effectiveness — Scientifically 
Based Research 

When school staff members meet to address school improvement 
efforts, they should first determine their needs and then decide which 
intervention, program, or model may best address those needs. 

Under NCLB legislation, funded schools must implement reform 
strategies informed by “scientifically based research” (SBR), 
which is defined in Title IX as having six specific components. To 
make these concepts more readily understood, we are providing 
more detailed explanations and examples to illustrate the six 
components. SBR must: 

• Use empirical methods. 

Quality research is conducted in a systematic and consistent 
manner with great attention to detail. The methods are appropriate, 
and conclusions are based on systematic observation or experimenl 

Practical example: You are the principal of an elementary school 
serving 500 students. In Grades 1-3, 70 percent of your students are 
bilingual and not achieving to grade-level standards. You are considering 
adopting a literacy model that claims to benefit all students, including bilingual students. Before adopting a 
new program, you should ask some questions about the research: Was this program tested under controlled 
circumstances in which some schools used the program and some did not? Was the sample size reasonable 
and relevant to the service provided? 

• Involve rigorous and adequate data analyses. 

Data collected in a high-quality study should be examined with appropriate statistical measurements to test 
the stated hypothesis and justify the conclusions drawn from the research. Failure to use proper statistical 
measurements could result in inaccurate or misleading results. 

Practical example: You are reviewing a study related to the effectiveness of a K-6 mathematics curriculum 
that claims significant increases in student achievement over a comparable program. Does the analysis 
account for students who have not stayed in the program for the length of the study? If yes, does it address 
how the data change as a result of students dropping out? Was the same measurement (test or assessment 
tool) used over time, and are comparisons and interpretations really possible given the tools used? 

• Rely on measurements or observational methods that provide reliable and valid data. 

Quality data produce accurate and credible findings. SBR utilizes measurements that provide reliable, valid 
data across multiple measurements and observations. Reliable instruments allow repeated measurements 
to produce similar results over time. Valid data will show that the assessment tool measures only what it 
is designed to measure. There must be a connection between the research question and the observed 
behavior; the research findings are based on that connection. Repeated measurements should yield 
identical or similar results from one evaluation to the next. 




Practical example: You are an elementary school reading specialist looking to improve reading 
comprehension. A professional development provider shows evidence that in a particular school that 
adopted its literacy approach, scores increased on a state assessment of third graders. You may reasonably 
have some concerns about bias, because the service provider may have selected the success stories. You 
might also have concerns about validity, because the test measures several aspects of reading performance 
in addition to comprehension. 



• Use either an experimental or quasi-experimental design. 

Quality studies use experimental research designs in which there is random assignment into at least two 
equivalent groups. One group participates in the program (receives the treatment), and the other group 
does not. If random assignment is not an option, quasi-experimental designs allow researchers to match the 
experimental and control groups as closely as possible. 

Practical example: You are a concerned parent and member of the school improvement planning team. 

) An external provider’s Web site reports the following: “In 20 schools that have used the MathStar approach, 
student performance on standardized tests increased by over 15 percent in three years compared to national 
norms!” Was there a control group? If so, were students assigned randomly? Did similar students in the area 
also make gains in mathematics performance over the same period? 

• Allow for replicability. 

Quality studies include a clear description of the method, instruments, and data used — a description 
sufficiently detailed to allow a different researcher to conduct the same study. Findings should be free 
from jargon to allow an informed lay reader to understand them. 

Practical example: A research study in a popular journal looked at the achievement of at-risk students in 
a computer-mediated environment. Does the study allow the reader to actively “participate” in the research 
process? Can the research process be recreated in another classroom with the same setting? Does the study 
provide enough information to carry out the experiment? 

• Undergo expert scrutiny. 

Quality studies have been reviewed and approved by independent expert reviewers. A form of quality control 
has been executed through expert analysis via rigorous, objective, scientific review of the research. 

Practical example: A teacher conducted a detailed case study of her experiences in implementing a new 
reading curriculum. The editor of a popular magazine that addresses a variety of educational issues decided 
that the experience was interesting and informative, so it was published. Would you consider this to be a 
highly scrutinized study? 




Evidence of Effectiveness — Cumulative 
Evidence Base 



School leaders must make decisions based on the best available evidence, 
as well as professional judgment, when implementing their programs (U.S. 
Department of Education, 2002). School reform programs, and the strategies 
and methods that make up programs, must reflect SBR (as defined above). 



It is important to remember that one study, even if it meets all the criteria of 
SBR, is not enough to be considered sufficient evidence of effectiveness. In 
most cases, a mix of research exists composed of numerous studies using 
various methodologies, including but not necessarily limited to experimental 
or quasi-experimental designs. If an experimental or quasi-experimental design 
has not been used, however, claims of causality are more tenuous and should be 
treated with caution. For a thorough but nontechnical guide to assessing the quality 
of research studies, see Identifying and Implementing Educational Practices Supported 
by Rigorous Evidence: A User Friendly Guide (U.S. Department of Education, 2003). 



It might be helpful to note that research exists on a continuum — from research that is purely theoretical in 
nature, to research that shows a correlation between variables, to research that shows causation. As shown 
in the diagram, an educational reform program typically starts out as a theoretical relationship between the 



variables of interest, is then subjected to correlational studies to understand the relationships of the variables, 
and finally is subjected to causal studies to examine whether the variables in question are actually causing, for 
example, changes in student achievement. 
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Progression from theory to scientifically based practice 



While this process does not always happen in the “real world,” when you are examining research 
that purports to make causal claims, look for evidence of this process. Because a 
program must adapt to meet the specific needs of a particular student population, 
it is important that clear evidence explains why and how a program works, not 
just that it does. For answers to the “why” or “how” questions, there must be a 
theory underlying the program, and evidence that previous research found that 
the theory successfully predicts certain outcomes. 

In order to meet NCLB criteria, school leaders must find ample research evidence 
for their reform program of choice. Finding SBR and building a cumulative 
research base are certainly daunting tasks, but resources are available to help. 

Some are listed in the next section. 



REFERENCES 



Cooper, H., & Valentine, J. C. (2003). What Works Clearinghouse cumulative research evidence 

assessment device (Version 0.6). Washington, DC: U.S. Department of Education. Retrieved 
December 10, 2003, from http://www.w-w-c.org/creadv06.doc 




No Child Left Behind Act of 2001, Pub. L. No. 107-110, 115 Stat. 1425 (2002). Retrieved December 10, 
2003, from http://www.ed.gov/policy/elsec/leg/esea02/index.html 

U.S. Department of Education. (2002). Comprehensive School Reform (CSR) guidance. Retrieved 
December 10, 2003, from http://www.ed.gov/programs/compreform/guidance/index.html 

U.S. Department of Education. (2003). Identifying and implementing educational practices supported by 
rigorous evidence: A user friendly guide. Washington, DC: Author. Retrieved January 8, 2004, 
from http://www.ed.gov/rschstat/research/pubs/rigorousevid/index.html 

Valentine, J. C., & Cooper, H. (2003). What Works Clearinghouse study design and implementation 
assessment device (Version 1.0). Washington, DC: U.S. Department of Education. Retrieved 
December 10, 2003, from http://www.w-w-c.org/DIAD_Final.doc 



LEARNING POINT ASSOCIATES KEY RESOURCES 



Learning Point Associates has resources to assist you in understanding and implementing NCLB in your 
school and district. Visit our Web site (www.learningpt.org) and Ahead of the Curve Web site (www.ncrel.org/ 
policy/curve) containing NCLB information. 

NCREL’s Learning Point magazine article “Wake-Up Call” (www.ncrel.org/info/nlp/lpsp03/wakeup.htm) 
outlines the challenges of implementing SBR in schools. 

NCREL’s Learning Point magazine article “Digging Out” (www.ncrel.org/info/nlp/lpsp03/dig.htm) focuses 
on using the best strategies to locate and identify SBR. 

Making Good Choices: A Guide for Schools and Districts (www.ncrel.org/csri/tools/makegood/title.htm), a 
revised publication, provides a process for examining school needs for reform and creating a reform initiative 
that will address those needs. 

Pathways to School Improvement Web site (www.ncrel.org/pathways) is designed primarily to help school 
improvement teams as they go through the phases within the school improvement process. 

ADDITIONAL KEY RESOURCES 

Scientifically Based Research and the CSR Program ( Comprehensive School Reform [CSR] Guidance: 
Appendix 0 (www.ed.gov/programs/compreform/guidance/appendc.pdf) is designed to help school staffs 
increase their understanding of what SBR is, and use that understanding to assess the quality, relevance, 
and usefulness of the research they examine. 

What Works Clearinghouse (www.w-w-c.org) Web site contains Web-based databases that provide a 
compendium of high-quality scientific research studies and reviews of scientific evidence of effectiveness 
of educational programs, practices, products, and policies. 

Identifying Research-Based Solutions for School Improvement (www.goodschools.gwu.edu/pubs/ 
sbrsub.php), an online workshop from the National Clearinghouse for Comprehensive School Reform, is 
designed to provide educators with the skills they need to find, identify, and make good use of the best 
available educational research. 

The Iowa Professional Development Model (www.state.ia.us/educate/ecese/tqt/tc/doc/ipdm02.pdf) 
provides a framework intended to assist districts, schools, and individuals as they design professional 
development programs. 

Education Commission of the States Web site (www.ecs.org) contains resources and a Web-based 
summary of state policies and activities related to NCLB requirements and SBR. 
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Glossary of Common Research Terms 

case study: An extensive study of an individual unit, group, institution, organization, or program. It provides 
a thorough analysis of the object being studied. 

causality: The relationship between the cause and the effect. Causality exists under three conditions: the 
intervention (e.g., a reform model) must come before the outcome (e.g., increased student achievement); 
there must be covariance (e.g., if the intervention increases slightly, the outcome will increase slightly); and 
there must not be extraneous explanations for the observed outcome. 

control group: A group of individuals whose characteristics are similar to an experimental group but who do 
not receive any of the program services or products being evaluated. 

correlation: A standardized measure of linear association between two variables. Its values range from -1 
(strong negative association) through zero (no association) to +1 (strong positive association). 

empirical research: Research conducted for the purpose of collecting measurable data in terms of attitudes, 
behavior, or performance. Designed to generate projectable, numerical data on a topic. 

evidence of effectiveness: A term used in the NCLB legislation, which states that programs must “prove 
evidence of effectiveness,” meaning a program must show — through student achievement data — that it 
improves achievement. 

experimental group: A group in an experimental design that receives the treatment or program. 

experimental research: A research design that involves random assignment of study participants to either 
an experimental or control group. This allows researchers to compare the outcome (e.g., test scores) of the 
experimental group to that of the control group, and to assess the effect of the treatment. The strongest 
research design for establishing evidence of effectiveness. 

qualitative research: Collection of nonnumerical data using interviews, observations, and open-ended 
questions to gather meaning from nonquantified narrative information. 

quantitative research: Collection of numerical data in order to describe, explain, predict, and/or control 
phenomena of interest. 

quasi-experimental research: A research design that includes some type of intervention or treatment and 
provides a comparison. Lacks the degree of control over all elements (environment, intervention, subject 
selection) found in experimental research design, but may provide some evidence of effectiveness. 
Comparison groups are frequently used, but maximum controls are employed to minimize threats to validity; 
random selection is typically not possible or practical. 

reliability: The degree to which an instrument consistently measures in the same way on repeated trials 
(e.g., a math test given to a student one day would yield roughly the same score if given to the same student 
the next day). 

theory: A comprehensive explanation of a given set of data that has been repeatedly confirmed by observa- 
tion and experimentation, and has gained general acceptance within the scientific community, but has not 
yet been decisively proven. 

validity of an instrument: The degree to which a measure accurately assesses the specific concept it is 
designed to measure (e.g., whether a reading-comprehension assessment focuses on students’ understand- 
ing of a story or their ability to read the story). 

validity of a research study: A study has internal validity when the possibility is minimal that other 
(confounding) variables are responsible for the effect in question. External validity is the extent to which 
the results of a study are generalizable and transferable to different populations, settings, and conditions. 
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